Apparatus for encoding image, apparatus for decoding image and image sensor

ABSTRACT

An image encoding apparatus includes a compressor that generates a bitstream including encoded data corresponding to values of original pixels, based on one of encoding modes, and a reconstructor that generates values of reference pixels by reconstructing the bitstream. In a first mode of the encoding modes, the compressor generates the bitstream based on a difference value between each of the values of the original pixels and a reference value which is based on at least one of the values of the reference pixels. In a second mode of the encoding modes, the compressor generates the bitstream based on an average value of at least two of the values of the original pixels.

CROSS-REFERENCE TO RELATED APPLICATIONS

This U. S. non-provisional application claims the benefit of priorityunder 35 U.S.C. § 119 to Korean Patent Application No. 10-2018-0063230filed on Jun. 1, 2018, in the Korean Intellectual Property Office(KIPO), the disclosure of which is incorporated by reference herein intheir entireties.

BACKGROUND

Various example embodiments of the inventive concepts described hereinrelate to an electronic apparatus, system, and/or method, and moreparticularly, relate to an apparatus, system, and/or method for encodingan image and an apparatus, system, and/or method for decoding an image.

Image encoding and/or image compression may refer to the process ofgenerating encoded image data, the size of which is smaller than that ofthe original image data, from the original image data. Also, imagedecoding and/or image decompression may refer to the process ofgenerating reconstructed image data by decoding the encoded image dataand/or a bitstream. The reconstructed image data may be the same as, ordifferent from, the original image data depending on the encoding anddecoding scheme.

Nowadays, as dual cameras are mounted on various electronic apparatusesand/or the number of images which may be captured per second increases,the size of image data which are stored in the electronic apparatuses isincreasing.

SUMMARY

Various example embodiments of the inventive concepts provide anapparatus, system, and/or method for encoding an original image of aBayer pattern, an apparatus, system, and/or method for decoding theencoded image, and/or an image sensor.

The technical problems to be solved by various example embodiments ofthe inventive concepts are not limited to the above-described technicalproblems, and other technical problems can be deduced from the followingexample embodiments.

According to at least one example embodiment, an image encodingapparatus may include a at least one processor configured to receive oneor more original pixels, and generate a compressed bitstream includingencoded data corresponding to values of the one or more original pixels,based on an encoding mode selected from a plurality of encoding modes,the generating including generating the compressed bitstream based on adifference value between each of the values of the original pixels and areference value, the reference value is based on at least one of thevalues of one or more reference pixels previously encoded andreconstructed, in response to the encoding mode being a first mode ofthe plurality of encoding modes, and generating the compressed bitstreambased on an average value of at least two of the values of the originalpixels in response to the encoding mode being a second mode of theplurality of encoding modes.

According to at least one example embodiment, an image decodingapparatus may include at least one processor configured to receive abitstream including an encoding mode selected from a plurality ofencoding modes, and generate values of one or more reconstruction pixelscorresponding to values of one or more original pixels based on thereceived bitstream, the generating including, decoding the bitstreambased on a difference value included in the bitstream and a referencevalue, the reference value is based on at least one value of one or morepreviously decoded reference pixels, in response to the encoding modebeing a first mode of the plurality of encoding modes, decoding thebitstream based on an average value received from the bitstream inresponse to the encoding mode being a second mode of the plurality ofencoding modes, and the difference value indicates a difference betweeneach of the values of the one or more original pixels and the referencevalue, and the average value is a value obtained by averaging at leasttwo of the values of the one or more original pixels.

According to an exemplary embodiment, an image sensor connected with anmemory may include an encoder that generates encoded data correspondingto values of original pixels of a Bayer color pattern based on one ofencoding modes, a memory controller that controls an operation ofinputting and outputting the encoded data to and from the memory, and adecoder that generates values of reconstruction pixels corresponding tothe values of the original pixels by decoding the encoded data outputfrom the memory. In a first mode of the encoding modes, the encoder maygenerate the encoded data based on a difference value between each ofthe values of the original pixels and a reference value which is basedon at least one of the values of the reference pixels previously encodedand reconstructed. In a second mode of the encoding modes, the encodermay generate the encoded data based on an average value of at least twoof the values of the original pixels.

BRIEF DESCRIPTION OF THE FIGURES

The above and other objects and features of the inventive concepts willbecome apparent by describing in detail various example embodimentsthereof with reference to the accompanying drawings.

FIG. 1 is a block diagram illustrating an image processing apparatusaccording to at least one example embodiment.

FIG. 2 is a diagram illustrating an image of a Bayer pattern obtainedthrough a Bayer color filter, according to at least one exampleembodiment.

FIG. 3 is a block diagram illustrating an encoder according to at leastone example embodiment.

FIG. 4 is a diagram illustrating an operation in which an encoderencodes pieces of original pixel data based on a DPCM (DifferentialPulse Code Modulation) mode according to at least one exampleembodiment.

FIG. 5A is a diagram illustrating an operation in which an encoder ofFIG. 3 encodes pieces of original pixel data based on an average mode,according to at least one example embodiment.

FIG. 5B is a diagram illustrating an operation in which an encoder ofFIG. 3 encodes pieces of original pixel data based on an average mode,according to at least one other example embodiment.

FIG. 5C is a diagram illustrating an operation in which an encoder ofFIG. 3 encodes pieces of original pixel data based on an average mode,according to at least one other example embodiment.

FIG. 6 is a table illustrating information about encoding modes to beused in an image processing apparatus of FIG. 1, according to at leastone example embodiment.

FIG. 7 is a flowchart illustrating a method in which an encoder of FIG.3 determines an encoding mode for pieces of original pixel data amongencoding modes illustrated in FIG. 6, according to at least one exampleembodiment.

FIG. 8 is a block diagram illustrating a decoder according to at leastone example embodiment.

FIG. 9 is a diagram illustrating an operation in which a decoder of FIG.8 decodes pieces of original pixel data based on a DPCM mode, accordingto at least one example embodiment.

FIG. 10 is a diagram illustrating an operation in which a decoder ofFIG. 8 decodes pieces of original pixel data based on an average mode,according to at least one example embodiment.

FIG. 11 is a diagram illustrating an operation in which a decoder ofFIG. 8 decodes pieces of original pixel data based on an average mode,according to at least one other example embodiment.

FIG. 12 is a block diagram illustrating an image processing apparatusaccording to at least one example embodiment.

FIG. 13 is a block diagram illustrating an image processing apparatusaccording to at least one example embodiment.

DETAILED DESCRIPTION

Below, various example embodiments of the inventive concepts will bedescribed in detail and clearly to such an extent that those(hereinafter referred to as “ordinary those”) skilled in the art mayimplement one or more of the inventive concepts.

FIG. 1 is a block diagram illustrating an image processing apparatusaccording to at least one example embodiment. FIG. 2 is a diagramillustrating an image of a Bayer pattern obtained through a Bayer colorfilter, according to at least one example embodiment.

An image processing apparatus 1000 is an electronic apparatus which maycapture and/or store an image associated with at least one subject byusing a solid state image sensor such as a complementary metal-oxidesemiconductor (CMOS) image sensor, etc. The image processing apparatus1000 may be a digital camera, a digital camcorder, a mobile phone,and/or a tablet computer, etc., but is not limited thereto.

Referring to FIG. 1, the image processing apparatus 1000 may include aBayer color filter 1200, an encoder 1400, a decoder 1600, and/or anapplication processor 1800, etc., but is not limited thereto. Modulesillustrated in FIG. 1 may be implemented with one processor or aplurality of processors, one or more processors including a plurality ofprocessing cores, application specific processing devices (e.g., ASICs,FPGAs, etc.), and/or may be independently implemented with a pluralityof different types of processors.

The Bayer color filter 1200 may obtain pieces of original pixel datafrom a light signal. The pieces of original pixel data may include apixel value of an original pixel. A Bayer pattern depends on thecondition that an eye of a person deduces most of lumiance data from agreen component of a subject. For example, 50% of pixels included in theBayer color filter 1200 may detect a green signal, 25% of the pixels maydetect a red signal, and 25% of the pixels may detect a blue signal.According to at least one example embodiment, the Bayer color filter1200 may have a configuration in which, for example, 2-by-2 cellpatterns each including a red (R) pixel, a blue (B) pixel, and two green(G) pixels are repeatedly arranged, but the example embodiments are notlimited thereto, for example the cell patterns may be a different sizedcell pattern. According to another example embodiment, the Bayer colorfilter 1200 may have a configuration in which, for example, 2-by-2 cellpatterns each including a red pixel, a blue pixel, and two wide green(W) pixels is repeatedly arranged, but the example embodiments are notlimited thereto.

The encoder 1400 may compress pieces of original pixel data obtainedthrough the Bayer color filter 1200, thereby reducing the size of imagedata. Referring to FIG. 2, a Bayer image 2000 indicates pieces oforiginal pixel data of a Bayer pattern obtained through the Bayer colorfilter 1200 according to at least one example embodiment. According toat least one example embodiment, the encoder 1400 may generate encodeddata associated with the pieces of original pixel data 2200 by usingpieces of reference pixel data 2400. Additionally, the encoder 1400 maygenerate the encoded data by using an average value of the pieces oforiginal pixel data 2200 without using the pieces of reference pixeldata 2400.

The encoder 1400 may determine pieces of original pixel data, which aresuccessive as much as the reference number, as one encoding unit (and/orchannel), like the pieces of original pixel data 2200 of FIG. 2. Piecesof original pixel data included in one encoding unit may be encoded to abitstream which is composed of a mode region for storing informationabout an encoding mode and/or a pixel region for storing encoded dataassociated with a value of original pixels, but is not limited thereto.

An operation of the encoder 1400 will be more fully described withreference to FIGS. 3 to 7 according to some example embodiments.

Returning to FIG. 1, the decoder 1600 may receive encoded data (i.e., abitstream) generated by the encoder 1400 and may generate decoded databy performing at least one decoding operation on the received encodingdata. The decoder 1600 may transfer the decoded data to the applicationprocessor 1800, but is not limited thereto. An operation of the decoder1600 will be more fully described with reference to FIGS. 8 to 11.

The application processor 1800 may be a central processing unit (CPU), amicroprocessor, and/or a micro controller unit (MCU), that has beenspecially configured and/or is specially programmed to perform postprocessing on the decoded data. The application processor 1800 mayperform post processing on pieces of decoded data received from thedecoder 1600. The post processing may mean applying an image enhancementalgorithm to image artifacts, etc., but is not limited thereto. Forexample, the application processor 1800 may perform, but is not limitedto, white balancing, denoising, demosaicking, lens shading, and/or gammacorrections, etc., on the received decoded data.

Below, the term “pixel” or “pixel value” may mean information and/or avalue output and/or obtained from a light signal through pixel elementsconstituting the Bayer color filter 1200. Below, the term “originalpixels” may mean pixel values of unit original pixels to be currentlyencoded and/or decoded. Below, the term “reference pixels” may meanpixel values of pixels to be referenced for the encoding of originalpixels. Below, the term “reconstruction pixels” may mean pixel values ofreconstruction pixels generated by decoding encoded data associated withoriginal pixels.

FIG. 3 is a block diagram illustrating an encoder according to at leastone example embodiment.

An encoder 3000 may output a bitstream including encoded data bycompressing received original pixels. The encoder 3000 corresponds tothe at least one example embodiment of the encoder 1400 of FIG. 1.Accordingly, even though omitted below, the above description given withregard to the encoder 1400 of FIG. 1 may be applied to the encoder 3000of FIG. 3.

Referring to FIG. 3, the encoder 3000 may include a bad pixel detector3200, a compressor 3400, a reconstructor 3600, and/or a reference pixelbuffer 3800, but the example embodiments are not limited thereto. Theencoder 3000 may include a central processor (not illustrated) and/orother special purpose processor which controls the bad pixel detector3200, the compressor 3400, the reconstructor 3600, and/or the referencepixel buffer 3800, etc., overall. Additionally, each of the bad pixeldetector 3200, the compressor 3400, the reconstructor 3600, and/or thereference pixel buffer 3800, etc., may be operated by its own processor(not illustrated), and the encoder 3000 may be overall operated as theprocessors (not illustrated) operate mutually and/or in a parallel ordistributed manner. Additionally, the bad pixel detector 3200, thecompressor 3400, the reconstructor 3600, and/or the reference pixelbuffer 3800, etc., may be controlled under control of an externalprocessor (not illustrated) of the encoder 3000.

The Bad pixel detector 3200 may detect a bad pixel which is present inone or more pieces of pixel data obtained by the Bayer color filter1200. For example, the bad pixel may include a static bad pixel causedby a physical error existing at a specific location of the Bayer colorfilter 1200 and/or a dynamic bad pixel that was caused irregularlyand/or intermittently. According to at least one example embodiment, thebad pixel detector 3200 may compare signal levels of a plurality ofpixels arranged in a horizontal direction with respect to a pixel to betested and may determine whether the pixel targeted for a test operationis included in an edge region of the whole image; in the case where thepixel to be tested is not included in the edge region, the bad pixeldetector 3200 may determine whether the pixel to be tested is defective,or in other words, whether the pixel value output by a correspondingpixel of the Bayer color filter is inaccurate and/or producing improperpixel values. According to at least one example embodiment, a bad pixelmay be detected by comparing signal levels of surrounding pixels of thetarget pixel. The bad pixel detector 3200 may tag attribute information(e.g., a flag, metadata, etc.) indicating a bad pixel to pixels whichare determined as a bad pixel.

The compressor 3400 may perform encoding (e.g., compression) on originalpixels, but is not limited thereto. According to at least one exampleembodiment, original pixels received by the compressor 3400 may bepixels which are marked by the bad pixel detector 3200 so as to indicatewhether original pixels are bad pixels (e.g., pixels that). Thecompressor 3400 may generate a bitstream (e.g., a compressed bitstream)including encoded data of an original pixel based on one of a pluralityof encoding modes. The encoding modes may include a differential pulsecode modulation (DPCM) mode in which encoding is performed based on adifference value between a value of each of the original pixels and areference value determined from one reference pixel; and an average modein which encoding is performed based on an average value of values ofthe original pixels. The reference pixel used in the DPCM mode may referto a previously encoded original pixel, and information about thereference pixel may be read from the reference pixel buffer 3800. Theencoding operation of the compressor 3400 will be more fully describedwith reference to FIGS. 4 to 7.

According to at least one example embodiment, the reconstructor 3600 maygenerate reference pixels by reconstructing the encoded data output fromthe compressor 3400. The expression “to refer to a previously encodedpixel” may mean to use a decoded pixel generated from an encodedoriginal pixel, and not to use the original pixel itself as a referencepixel. With regard to the original pixel, information about an originalpixel and information about an encoded pixel and a decoded pixel from anoriginal pixel may be present in the encoder 3000, but a decoder (e.g.,1600 of FIG. 1, etc.) does not/cannot know information about theoriginal pixel. That is, in the case where the encoder 3000 uses aprevious original pixel as a reference pixel, since the decoder cannotknow information about the previous original pixel referenced by theencoder 3000, the results of reconstruction of the original pixel mayvary, and a mismatch may occur between the encoder 3000 and the decoder.Accordingly, the encoder 3000 should use a pixel, which is againreconstructed from previously encoded original pixels, as a referencepixel.

The reconstructor 3600 may store the reconstructed pixel to a memory(not illustrated). The memory (not illustrated) may include a volatilememory such as a dynamic random access memory (DRAM) and/or a static RAM(SRAM), etc., and/or a nonvolatile memory such as a phase-change randomaccess memory (PRAM), a magnetic random access memory (MRAM), aresistive RAM (ReRAM), and/or a ferroelectric RAM (FRAM), etc.

The reference pixel buffer 3800 may extract information about at leastone reference pixel for encoding of current original pixels from thememory (not illustrated). A reference pixel may be neighborhood pixelsof original pixels, or in other words, the one or more reference pixelsmay be one or pixels neighboring the one or more original pixels.According to at least one example embodiment, the reference pixel buffer3800 may be composed of line memories (e.g., lines of memories) forstoring pixel values of neighborhood pixels of original pixels desiredand/or necessary for encoding of the original pixels. The referencepixel buffer 3800 according to at least one example embodiment may beimplemented with, but is not limited to, a volatile memory such as aDRAM and/or an SRAM.

Below, operations of the encoder 3000 will be more fully described withreference to FIGS. 4 to 7C. Operations to be described with reference toFIGS. 4 to 7C will be performed by the compressor 3400, but is notlimited thereto.

FIG. 4 is a diagram illustrating an operation in which an encoder ofFIG. 3 encodes pieces of original pixel data based on a DPCM modeaccording to at least one example embodiment.

In the DPCM mode, the encoder 3000 may encode original pixels 4200 basedon a reference value determined from at least one reference pixel, butis not limited thereto. According to at least one example embodiment,pixels positioned on, for example, the upper two lines of the originalpixels 4200 may be used as reference pixels, however the exampleembodiments are not limited thereto, and other pixels may be used as thereference pixels. The original pixels 4200 may include a first originalpixel G_(T0) being a green pixel, a second original pixel R_(T0) being ared pixel, a third original pixel G_(T1) being a green pixel, and afourth original pixel R_(T1) being a red pixel, but are not limitedthereto. According to another example embodiment, the second originalpixel R_(T0) and the fourth original pixel R_(T1) may be a blue pixel,etc. However, for convenience of description, it is assumed that thesecond original pixel R_(T0) and the fourth original pixel R_(T1) are ared pixel. The original pixels 4200 may be pixels before compression,but are not limited thereto. For example, a pixel value of each of theoriginal pixels 4200 may be expressed by a value which is equal to orgreater than “0” and is smaller than “1024”.

Referring to a bitstream 4800 generated by the encoder 3000 by encodingthe original pixels 4200, the bitstream 4800 may include a mode regionfor storing information (e.g., a bit string indicating the DPCM mode,etc.) about an encoding mode of the encoder 3000 used to generate thebitstream 4800, and pixel regions, e.g., DPCM1 to DMCM4, etc., forstoring information about a pixel value related to pixels correspondingto the original pixels 4200. For example, the bitstream 4800 may becomposed of a first pixel region DPCM1 for storing encoded dataassociated with the first original pixel G_(T0), a second pixel regionDPCM2 for storing encoded data associated with the second original pixelR_(T0), a third pixel region DPCM3 for storing encoded data associatedwith the third original pixel G_(T1), and a fourth pixel region DPCM4for storing encoded data associated with the fourth original pixelR_(T1), etc.

The mode region Mode may include M-bit data. Each of the first pixelregion DPCM1, the second pixel region DPCM2, the third pixel regionDPCM3, and the fourth pixel region DPCM4 may include N-bit data. “M” and“N” may be positive integers which are greater than “0”. Below, it isassumed that “M” and “N” are “4”, but the example embodiments are notlimited thereto.

The encoder 3000 may store a difference value d1 between a pixel valueof the first original pixel G_(T0) and a reference value to the firstpixel region DPCM1. According to at least one example embodiment, anaverage value of a pixel value of a reference pixel G0 positioned on afirst location, e.g., the upper left side of the first original pixelG_(T0), and a pixel value of a reference pixel G1 positioned on a secondlocation, e.g., the upper right side thereof, may be used as a referencevalue, but the example embodiments are not limited thereto. That is, thedifference value d1 stored to the first pixel region DPCM1 may bedetermined by the following Equation 1.d1=(G0+G1)/2−G _(T0)  [Equation 1]

The encoder 3000 may store a difference value d3 between a pixel valueof the second original pixel R_(T0) and a reference value to the secondpixel region DPCM2. According to at least one example embodiment, apixel value of a reference pixel RB1 positioned on, e.g., the secondline, above the second original pixel R_(T0) may be used as a referencevalue. That is, the difference value d3 stored to the second pixelregion DPCM2 may be determined by the following Equation 2.d3=RB1−R _(T0)  [Equation 2]

The encoder 3000 may store a difference value d2 between a pixel valueof the third original pixel G_(T1) and a reference value. According toat least one example embodiment, an average value between a pixel valueof a reference pixel G1 positioned on a first location, e.g., the upperleft side of the third original pixel G_(T1), and a pixel value of areference pixel G2 positioned on a second location, e.g., the upperright side of the third original pixel G_(T1), may be used as areference value. That is, the difference value d2 stored to the thirdpixel region DPCM3 may be determined by the following Equation 3.d2=(G1+G2)/2−G _(T1)  [Equation 3]

The encoder 3000 may store a difference value d4 between a pixel valueof the fourth original pixel R_(T1) and a reference value to the fourthpixel region DPCM4. According to at least one example embodiment, apixel value of a reference pixel RB2 positioned on the second line abovethe fourth original pixel R_(T1) may be used as a reference value. Thatis, the difference value d4 stored to the fourth pixel region DPCM4 maybe determined by the following Equation 4.d4=RB2−R _(T1)  [Equation 4]

Since each of the difference values, e.g., d1, d2, d3, and d4, etc.,determined by Equation 1 to Equation 4 may be a negative value or apositive value, the most significant bit of each of the pixel regionsDPCM1, DPCM2, DPCM3, and DPCM4 may be a sign bit indicating signinformation. For example, in the case where the difference value d1determined by Equation 1 is “−6”, a bit string of “1110” including asign bit of “1” indicating a negative value and a bit string of “110”indicating “6” may be stored to the first pixel region DPCM1.

Additionally, the encoder 3000 according to at least one exampleembodiment may perform a bit shift operation on each of the differencevalues d1, d2, d3, and d4, etc., determined by Equation 1 to Equation 4based on the size of a relevant pixel region. For example, in the casewhere the number of bits assigned to the first pixel region DPCM1 is “4”and the difference value d1 is 16₁₀(10000₂), the encoder 3000 may storea bit string of “100”, which is obtained by performing a right bit shiftoperation (“>>”) on the difference value d1 two times, to the firstpixel region DPCM1 together with the signal bit of “0”. The encoder 3000may store information about the number of times that a shift operationis performed, to the mode region.

However, positions of pixels to be referenced for encoding of originalpixels described with reference to FIG. 4 may be only one exampleembodiment and may be changed. For example, the encoder 3000 may not usereference pixels determined as a bad pixel by the bad pixel detector3200 of FIG. 3 for the purpose of determining a reference value.

In the case of encoding the first original pixel G_(T0), in the casewhere the reference pixel G0 is a bad pixel, the encoder 3000 maydetermine not an average value of the reference pixels G0 and G1, butrather a pixel value of the reference pixel G1 as the reference value.In the case where the reference pixels G0 and G1 are a bad pixel, apixel value of a reference pixel G5 may be determined as a referencevalue. In the case where reference pixels G0, G1, and G5 are bad pixels,reference values may be determined based on at least one pixel value ofreference pixels G4 and G6 positioned in a diagonal direction, etc.

In the case of encoding the second original pixel R_(T0), for example,in the case where the reference pixel RB1 is a bad pixel, the encoder3000 may sort reference pixels RB0, RB2, and RB3, etc., to select anintermediate value and may determine the selected intermediate value asa reference value.

FIG. 5A is a diagram illustrating an operation in which an encoder ofFIG. 3 encodes pieces of original pixel data based on a first averagemode, according to at least one example embodiment.

The encoder 3000 may encode original pixels 5220 based on a firstaverage mode and may generate a bitstream 5300, but is not limitedthereto.

Referring to the bitstream 5300, the bitstream 5300 may include a moderegion 5340 including information (e.g., a bit string indicating thefirst average mode, etc.) about an encoding mode, a first pixel region5360 for storing encoded data associated with the first original pixelG_(T0) and the third original pixel G_(T1) being a green pixel, and asecond pixel region 5380 for storing encoded data associated with thesecond original pixel R_(T0) and the fourth original pixel R_(T1) beinga red pixel, but the example embodiments are not limited thereto.

The encoder 3000 may store an average value of a pixel value of thefirst original pixel G_(T0) and a pixel value of the third originalpixel G_(T1) to the first pixel region 5360. The encoder 3000 mayperform a bit shift operation on the first average value depending on(and/or based on) the size of the first pixel region 5360. For example,in the case where the number of bits assigned to the first pixel region5360 is “8” and the first average value is 488₁₀(=111101000₂), theencoder 3000 may perform a right bit shift operation (“>>”) on the firstaverage value and may store “1111010” to the first pixel region 5360. Inthis case, since the pieces of encoded data associated with two originalpixels G_(T0) and G_(T1) are the same as each other, the encoded data ofthe first and third original pixels G_(T0) and G_(T1) may be storedtogether in the first pixel region 5360.

The encoder 3000 may store a second average value of a pixel value ofthe second original pixel R_(T0) and a pixel value of the fourthoriginal pixel R_(T1) to the second pixel region 5380. A bit shiftoperation may be performed on the second average value depending on thesize of the second pixel region 5380, identically to the way to storethe first average value to the first pixel region 5360.

The first average mode described with reference to FIG. 5A may beefficient when the upper side of the original pixels 5220 corresponds toa horizontal edge. In this case, since spatial correlation almost doesnot exist between reference pixels positioned on the upper side of theoriginal pixels 5220, it may be more efficient to perform encoding byusing the pixel values of the original pixels 5220 itself.

FIGS. 5B and 5C are diagrams illustrating an operation in which anencoder of FIG. 3 encodes pieces of original pixel data based on anaverage mode, according to another example embodiment.

FIG. 5B shows an operation in which the encoder 3000 encodes originalpixels 5420 based on a second average mode and generates a bitstream5500 according to at least one example embodiment, and FIG. 5C shows anoperation in which the encoder 3000 encodes original pixels 5620 basedon a third average mode and generates a bitstream 5700 according to atleast one example embodiment. The encoding based on the second averagemode and the third average mode may be especially efficient when ahorizontal edge exists at the upper side of original pixels and a pixelincluded in an edge region is also present in the original pixels.

Referring to FIG. 5B, the fourth original pixel RT₁ of the originalpixels 5420 may be a pixel included in an edge region. A bitstream 5500may include a mode region 5520 including information (e.g., a bit stringindicating the second average mode, etc.) about an encoding mode, afirst pixel region 5540 for storing encoded data associated with thefirst original pixel G_(T0) and the third original pixel G_(T1), asecond pixel region 5560 for storing encoded data associated with thesecond original pixel R_(T0), and a third pixel region 5580 for storingencoded data associated with the fourth original pixel R_(T1), but theexample embodiments are not limited thereto.

The encoder 3000 may store an average value of a pixel value of thefirst original pixel G_(T0) and a pixel value of the third originalpixel G_(T1) to the first pixel region 5540. However, since the fourthoriginal pixel R_(T1) corresponds to an edge component, spatialcorrelation almost does not exist between the second original pixelR_(T0) and the fourth original pixel R_(T1). Accordingly, in the casewhere an average value of the second original pixel R_(T0) and thefourth original pixel R_(T1) is used as encoded data, the informationloss may be great. Accordingly, the encoder 3000 may respectively storea pixel value of the second original pixel R_(T0) and a pixel value ofthe fourth original pixel R_(T1) to the second pixel region 5560 and thethird pixel region 5580 without separate conversion (e.g., a bit shiftoperation which is performed depending on the size of the second pixelregion 5560 and the third pixel region 5580 is excluded).

Referring to FIG. 5C, the first original pixel G_(T0) of the originalpixels 5620 may correspond to an edge component according to at leastone example embodiment. A bitstream 5700 may include a mode region 5720including information (e.g., a bit string indicating the third averagemode, etc.) about an encoding mode, a first pixel region 5740 forstoring encoded data associated with the first original pixel G_(T0), asecond pixel region 5760 for storing encoded data associated with thethird original pixel G_(T1), and a third pixel region 5780 for storingencoded data associated with the second original pixel R_(T0) and thefourth original pixel R_(T1).

A method for encoding the original pixels 5620 based on the thirdaverage mode is similar to the method for encoding the original pixels5420 based on the second average mode, which is described with referenceto FIG. 5B. The difference is as follows. Since an edge component of theoriginal pixels 5620 is the first original pixel G_(T0), a pixel valueof the first original pixel G_(T0) and a pixel value of the thirdoriginal pixel G_(T1) may be respectively stored to the first pixelregion 5740 and the second pixel region 5760 without separate conversion(e.g., a bit shift operation which is performed depending on the size ofthe first pixel region 5740 and the second pixel region 5760 isexcluded). For example, the encoder 3000 may store an average value of apixel value of the second original pixel R_(T0) and a pixel value of thefourth original pixel R_(T1) to the third pixel region 5780.

According to at least one example embodiment, the encoder 3000 mayhandle the second average mode described with reference to FIG. 5B andthe third average mode described with reference to FIG. 5C as the sameencoding mode and may include an additional bit for distinguishing thesecond average mode from the third average mode in the generatedbitstream. For example, the mode region 5520 of the bitstream 5500 andthe mode region 5720 of the bitstream 5700 may include an additionalregion for indicating sub mode information, but is not limited thereto.The encoder 3000 may distinguish the second average mode from the thirdaverage mode by recording a value, e.g., of “0” or “1” in the additionalregion. In this case, the size of each of pixel regions of the bitstream5500 and the bitstream 5700 may decrease due to the additional regionfor indicating the sub mode information.

FIG. 6 is a table illustrating information about encoding modes to beused in an image processing apparatus of FIG. 1, according to at leastone example embodiment.

Referring to a table 6000, encoding modes may include the DPCM modedescribed with reference to FIG. 4, the average mode described withreference to FIGS. 5A to 5C, and/or a pulse code modulation (PCM) mode,but the example embodiments are not limited thereto. For example, 13encoding modes, e.g., Mode 0 to Mode 12, are stored in a first columnMode of the table 6000, but the example embodiments are not limitedthereto. Encoding methods DPCM, Average, and/or PCM, etc., which areused in the 13 encoding modes are stored in a second column Method butthe example embodiments are not limited thereto. The number of timesthat a bit shift operation is performed to store a difference valueand/or an average value associated with a pixel value, e.g., a greenpixel, is stated at a third column Shift G. The number of times that abit shift operation is performed to store a difference value and/or anaverage value associated with a red or blue pixel is recorded at afourth column Shift RB, etc.

The table 6000 defines a protocol between the encoder 3000 and thedecoder 1600. That is, the table 6000 indicates information desiredand/or necessary for the decoder 1600 to generate a reconstruction pixelbased on a bitstream received from the encoder 1400. Pieces ofinformation recorded in the table 6000 may be experimentally obtainedvalues.

For example, the encoder 1400 may compress original pixels based on Mode4 of the table 6000 and may record “4” in a mode region of the generatedbitstream. The decoder 1600 may parse “4” from the mode region of thebitstream received from the encoder 1400. The decoder 1600 may determinethat the current original pixels are compressed in the DPCM mode basedon the values stored in Method column (e.g., category) of the table6000, and the decoder 1600 may determine that a difference value of agreen pixel stored in each of a first pixel region and a third pixelregion is a result value obtained by performing a bit shift operationthree times based on the Shift G column (e.g., category) of the table6000. Also, the decoder 1600 may determine that a difference value of ared/blue pixel stored in each of a second pixel region and a fourthpixel region is a result value obtained by performing a bit shiftoperation three times based on the Shift R/B column (e.g., category) ofthe table 6000.

For example, the encoder 1400 may compress the original pixels based onMode 10 of the table 6000 and may record “10” in a mode region of thegenerated bitstream. The decoder 1600 may determine that currentoriginal pixels are compressed in the average mode and an average valueof green pixels stored in the first pixel region is a result valueobtained by performing a bit shift operation two times from the table6000. Also, the decoder 1600 may determine that an average value of ared/blue pixel stored in the second pixel region is a result valueobtained by performing a bit shift operation two times.

According to at least one example embodiment, Mode 12 of the table 6000may indicate the PCM mode, but the example embodiments are not limitedthereto. The PCM mode may be a mode of including only the upper bitvalues in a bitstream suitably for the size of a relevant pixel region,without sequentially converting the pixel values of current originalpixels. The encoder 1400 according to at least one example embodimentmay attempt encoding by using Mode 0 to Mode 12 sequentially, and mayperform encoding based on the PCM mode when Mode 1 to Mode 12 all fail,but the example embodiments are not limited thereto.

FIG. 7 is a flowchart illustrating a method in which an encoder of FIG.3 determines an encoding mode for pieces of original pixel data amongencoding modes illustrated in FIG. 6, according to at least one exampleembodiment.

In operation S7100, the encoder 3000 may determine whether to encodeoriginal pixels based on the DPCM mode. For example, the encoder 3000may sequentially determine whether to encode original pixels based onvarious modes corresponding to DPCM, e.g., Mode 1 to Mode 9 in the table6000 of FIG. 6. That is, in the case where encoding is possible in acurrent mode (Yes), the encoder 3000 may determine a current DPCM mode“Mode n” (e.g., n being any one of 1 to 9) as an encoding mode; if not(No), the encoder 3000 may determine whether encoding is possible in anext DPCM mode. Since the number of times of a bit shift operationincreases when progressing toward Mode 9 from Mode 1, the loss oforiginal data due to compression may occur less when encoding isperformed in a mode having a smaller number (e.g., compression usingMode 1 may result in less loss/error than compression of original datausing Mode 9, etc.).

For example, the encoder 3000 may determine whether to encode theoriginal pixels based on Mode 0 in the table 6000 of FIG. 6. Referringto the table 6000, a bit shift operation is not performed in Mode 0.Accordingly, in the case where any one of the difference value d1associated with the first original pixel G_(T0), the difference value d2associated with the second original pixel R_(T0), the difference valued3 associated with the third original pixel G_(T1), and the differencevalue d4 associated with the fourth original pixel R_(T1), is notexpressed within a given bit set (e.g., four bits) set to the size of apixel region, the encoder 3000 may determine that it is difficult,undesirable, and/or impossible to encode the original pixels using Mode0. That is, in the case where it is determined that Mode 0 is usable,the encoder 3000 may determine Mode 0 as an encoding mode for theoriginal pixels; however, in the case where it is determined that Mode 0is unusable, the encoder 3000 may determine whether to use a next DPCMmode (i.e., Mode 1).

In the case where it is determined based on the DPCM mode that encodingof the original pixels is difficult, undesirable, and/or impossible(No), in operation S7300, the encoder 3000 may determine whether toencode the original pixels based on the average mode. For example, theencoder 3000 may sequentially determine whether to encode the originalpixels currently based on, e.g., Mode 10 and Mode 11, in the table 6000of FIG. 6. In the case where it is determined that encoding is currentlypossible in the average mode “Mode k” (e.g., k being 10 or 11, etc.), inoperation S7500, the encoder 3000 may perform an additional testoperation. However, in the case where encoding is difficult,undesirable, and/or impossible even though any mode of Mode 10 and Mode11 is used (No), in operation S7400, the encoder 3000 may determine thePCM mode as an encoding mode for the original pixels.

In operation S7500, the encoder 3000 may compare the degree of data lossdue to compression of the average mode “Mode k”, which is determined inoperation S73000 as encoding may be possible, and the degree of dataloss due to compression of the PCM mode. As described above, a bit shiftoperation may be performed depending on the size of a pixel regionassigned to a bitstream, and the encoder 3000 may compare the levels ofdata loss by comparing the number of times that a bit shift operation isperformed in the average mode “Mode k” and the number of times that abit shift operation is performed in the PCM mode. In the case where thelevel of data loss in the PCM mode is greater than the level of dataloss in the average mode “Mode k” determined in operation S7300 (Yes),the encoder 3000 may determine the average mode “Mode k” determined inoperation S7300 as an encoding mode for the original pixels (S7600). Inthe case where the level of data loss in the PCM mode is not greaterthan the level of data loss in the average mode Mode k determined inoperation S7300 (No), the encoder 3000 may determine the PCM mode as anencoding mode for the original pixels (S7400).

FIG. 8 is a block diagram illustrating a decoder according to at leastone example embodiment.

A decoder 8000 corresponds to a detailed example embodiment of thedecoder 1600 of FIG. 1, but is not limited thereto. Accordingly, eventhough omitted below, the above description given with regard to thedecoder 1600 of FIG. 1 may be applied to the decoder 8000 of FIG. 8,etc. It may be easily understood to ordinary those that an operationcorresponding to the encoder 3000 of FIG. 3 may be performed in thedecoder 8000.

Referring to FIG. 8, the decoder 8000 may include a bad pixel detector8200, a decompressor 8400, a reconstructor 8600, and/or a referencepixel buffer 8800, but is not limited thereto. The decoder 8000 mayinclude at least one central processor (not illustrated) which controlsthe bad pixel detector 8200, the decompressor 8400, the reconstructor8600, and/or the reference pixel buffer 8800, etc. Additionally, each ofthe bad pixel detector 8200, the decompressor 8400, the reconstructor8600, and/or the reference pixel buffer 8800 may be operated by theirown processor (not illustrated), and the decoder 8000 may be overalloperated as the processors (not illustrated) operate mutually and/or ina parallel or distributed manner. Additionally, the bad pixel detector8200, the decompressor 8400, the reconstructor 8600, and/or thereference pixel buffer 8800 may be controlled under the control of oneor more external processors (not illustrated) of the decoder 8000.

The bad pixel detector 8200, the reconstructor 8600, and/or thereference pixel buffer 8800 correspond to the bad pixel detector 3200,the reconstructor 3600, and/or the reference pixel buffer 3800 of FIG.3, and thus, additional description will be omitted to avoid redundancy.

The decompressor 8400 may determine an encoding mode from a bitstreamreceived from the encoder 3000 of FIG. 3, and may generate pieces ofreconstruction pixel data based on the determined encoding mode, but isnot limited thereto. According to at least one example embodiment, thedecompressor 8400 may parse information included in a mode region of thereceived bitstream and may determine an encoding mode for originalpixels based on the parsed information, but is not limited thereto.

Below, operations of the decoder 8000 will be more fully described withreference to FIGS. 9 to 11 according to at least one example embodiment.Operations to be described with reference to FIGS. 9 to 11 will beperformed by the decompressor 8400 according to at least one exampleembodiment.

FIG. 9 is a diagram illustrating an operation in which a decoder of FIG.8 decodes original pixels based on a DPCM mode, according to at leastone example embodiment.

The decoder 8000 may determine an encoding mode for original pixels asthe DPCM mode with reference to a mode region Mode of a receivedbitstream 9000. For example, it is assumed that an encoding mode isdetermined as Mode 1 of FIG. 6, but is not limited thereto.

The decoder 8000 may generate a first reconstruction pixel G_(R0)associated with, for example, the first original pixel G_(R0) by using afirst difference value d1 read from the first pixel region DPCM1 and areference value. In the DPCM mode, since a green pixel is encoded byusing, as a reference value, an average value of a pixel value of areference pixel positioned on a first location, e.g., the upper leftside of an original pixel, and a pixel value of a reference pixelpositioned on a second location, e.g., the upper right side thereof, thedecoder 8000 may read information about the reference pixel from thereference pixel buffer 8800 and may determine the reference value basedon the read reference pixel information corresponding to the firstlocation and the second location. That is, the decoder 8000 maydetermine a value of the first reconstruction pixel G_(R0) by using thefollowing Equation 5.G′R0=(G0+G1)/2−d1(G0 and G1: reference pixel)  [Equation 5]

The decoder 8000 may generate a reconstruction pixel R′_(R0) associatedwith the second original pixel R_(R0) by using the second differencevalue d2 read from the second pixel region DPCM2 and a reference value.In the DPCM mode, since a red or blue pixel is encoded by using a pixelvalue of a reference pixel positioned on, e.g., an upper two lines ofthe original pixel as a reference value, the decoder 8000 may readinformation about the reference pixel from the reference pixel buffer8800 and may determine the reference value based on the read referencepixel information. That is, the decoder 8000 may determine a value ofthe second reconstruction pixel R′_(R0) by using the following Equation6.R′R0=RB1−d2(RB1: reference pixel)  [Equation 6]

The decoder 8000 may respectively read the third difference value d3 andthe fourth difference value d4 from the third pixel region DPCM3 and thefourth pixel region DPCM4 and may generate a third reconstruction pixelG′_(R1) and a fourth reconstruction pixel R′_(R1) by using the readvalues and a reference value. A method in which the decoder 8000generates the third reconstruction pixel G′_(R1) and the fourthreconstruction pixel R′_(R1) is the same as the method in which thedecoder 8000 generates the first reconstruction pixel G′_(R0) and thesecond reconstruction pixel R′_(R0), and thus, additional descriptionwill be omitted to avoid redundancy.

According to at least one example embodiment, the decoder 8000 mayperform a bit shift operation, in an opposite direction to a bit shiftoperation which the encoder 3000 performs a bit shift operation on eachof the difference values d1, d2, d3, and d4, etc., read from pixelregions.

For example, in the case where an encoding mode is determined as Mode 3of FIG. 6, the decoder 8000 may perform a left bit shift operation(“<<”) (e.g., the opposite direction bit shift operation than the bitshift operation performed by the encoder) on the first difference valued1 read from the first pixel region DPCM1 and the third difference valued3 read from the third pixel region DPCM3 two times, and may generatethe first reconstruction pixel G_(R0) and the third reconstruction pixelG′_(R1) based on result values of the left bit shift operation. Thedecoder 8000 may perform a left bit shift operation (“<<”) on the seconddifference value d2 read from the second pixel region DPCM2 and thefourth difference value d4 read from the fourth pixel region DPCM4 threetimes and may generate the second reconstruction pixel R′_(R0) and thefourth reconstruction pixel R′_(R1) based on result values of the leftbit shift operation.

FIG. 10 is a diagram illustrating an operation in which a decoder ofFIG. 8 decodes original pixels based on an average mode, according to atleast one example embodiment.

The decoder 8000 may determine an encoding mode for original pixels asthe average mode with reference to a mode region 10020 of a receivedbitstream 10000, but is not limited thereto.

The decoder 8000 may determine a first average value d1 read from afirst pixel region 10040 as a pixel value of the first reconstructionpixel G_(R0) and the third reconstruction pixel G′_(R1).

The decoder 8000 may determine a second average value d2 read from asecond pixel region 10060 as a pixel value of the second reconstructionpixel R′_(R0) and the fourth reconstruction pixel R′_(R1).

As in the DPCM mode, the decoder 8000 according to at least one exampleembodiment may perform a left bit shift operation (“<<”) on each of thefirst average value d1 read from the first pixel region 10040 and thesecond average value d2 read from the second pixel region 10060, basedon information read from the mode region 10020.

For example, in the case where an encoding mode is determined as Mode 10of FIG. 6, the decoder 8000 may determine a value, which is obtained byperforming a left bit shift operation (“<<”) on the first average valued1 two times, as a pixel value of the first reconstruction pixel G_(R0)and the third reconstruction pixel G′_(R1), and may determine a value,which is obtained by performing a left bit shift operation (“<<”) on thesecond average value d2 two times, as a pixel value of the secondreconstruction pixel R′_(R0) and the fourth reconstruction pixelR′_(R1).

FIG. 11 is a diagram illustrating an operation in which a decoder ofFIG. 8 decodes original pixels based on an average mode, according toanother example embodiment.

The decoder 8000 may determine an encoding mode for original pixels asthe average mode with reference to a mode region 11020 of a receivedbitstream 11000. For example, it is assumed that an encoding mode isdetermined as Mode 11 of FIG. 6, but the example embodiments are notlimited thereto. According to at least one example embodiment, in thecase where an additional region Sub mode is included in the mode region11020 including mode information, the decoder 8000 may determine anencoding mode as Mode 11, etc.

As described with reference to FIGS. 5B and 5C, a decoding method may bedifferently determined depending (e.g., based) on a value read from theadditional region Sub mode. For example, in the case where a value readfrom the additional region Sub mode is “0” may indicate that an edgecomponent is present in a right pixel of original pixels, and the casewhere a value read from the additional region Sub mode is “1” mayindicate that an edge component is present in a left pixel of theoriginal pixels, but is not limited thereto.

According to at least one example embodiment, in the case where a valueread from the additional region Sub mode is “0”, depending on the numberof times of a bit shift operation defined in the table 6000 of FIG. 6,the decoder 8000 may determine a value, which is obtained by performinga left bit shift operation (“<<”) on a value d1 read from a first pixelregion 11040 five times, as a pixel value of the first reconstructionpixel G′_(R0) and a pixel value of the third reconstruction pixelG′_(R1). The decoder 8000 may determine a value, which is obtained byperforming a left bit shift operation (“<<”) on a value d2 read from asecond pixel region 11060 five times, as a pixel value of the secondreconstruction pixel R′_(R0). Also, the decoder 8000 may determine avalue, which is obtained by performing a left bit shift operation (“<<”)on a value d3 read from a third pixel region 11080 five times, as apixel value of the second reconstruction pixel R′_(R1).

According to at least one example embodiment, in the case where a valueread from the additional region Sub mode is “1”, depending on the numberof times of a bit shift operation defined in the table 6000 of FIG. 6,the decoder 8000 may determine a value, which is obtained by performinga left bit shift operation (“<<”) on the value d1 read from the firstpixel region 11040 five times, as a pixel value of the firstreconstruction pixel G′_(R0). The decoder 8000 may determine a value,which is obtained by performing a left bit shift operation (“<<”) on thevalue d2 read from the second pixel region 11060 five times, as a pixelvalue of the third reconstruction pixel G′_(R1). Also, the decoder 8000may determine a value, which is obtained by performing a left bit shiftoperation (“<<”) on the value d3 read from the third pixel region 11080five times, as a pixel value of the second reconstruction pixel R′_(R0)and a pixel value of the fourth reconstruction pixel R′_(R1).

FIG. 12 is a block diagram illustrating an image processing apparatusaccording to at least one example embodiment.

An image processing apparatus 12000 corresponds to a detailed exampleembodiment of the image processing apparatus 1000 of FIG. 1, but is notlimited thereto. Accordingly, even though omitted below, the abovedescription given with regard to the image processing apparatus 1000 ofFIG. 1 may be applied to the image processing apparatus 12000.

Referring to FIG. 12, the image processing apparatus 12000 may includean image sensor 12200, a memory 12400, and/or an application processor12600, etc., but the example embodiments are not limited thereto.

The image sensor 12200 may be an electronic part which extracts colorinformation from a light signal so as to be implemented in a display(e.g., a display panel, a display device, a projector, etc.). The imagesensor 12200 may include an encoder 12220, a decoder 12240, and/or amemory controller 12260, etc., but is not limited thereto. The imagesensor 12200 is also called a “3 stack sensor”.

The encoder 12220 may perform the operation of the encoder 1400described with reference to FIGS. 1 to 7 according to at least oneexample embodiment. That is, the encoder 12220 may encode originalpixels of a received Bayer pattern based on, for example, the DPCM modeand the average mode, etc., and may generate a bitstream includingencoded data. The bitstream generated by the encoder 12220 may beprovided to the memory controller 12260.

The memory controller 12260 may control an operation of inputting andoutputting the encoded data. The bitstream generated by the encoder12220 may be input to the memory 12400 under the control of the memorycontroller 12260. The memory controller 12260 may include a dedicatedlogic circuit (e.g., FPGA or ASICs, etc.) which performs variousoperations for controlling overall operations in the memory 12400.

The memory 12400 may be connected to the image sensor 12200 and maystore image frames. The memory 12400 may store not original dataassociated with image frames but pieces of encoded data generated by theencoder 12220. Accordingly, the number of image frames which may bestored to the memory 12400 may greatly increase compared to the case ofstoring original data to the memory 12400. The memory 12400 may includea volatile memory such as a DRAM and/or an SRAM, etc., and/or anonvolatile memory such as a flash memory, a PRAM, an MRAM, a ReRAM,and/or an FRAM, etc.

The memory 12400 may output a bitstream including encoded data to thedecoder 12240 under the control of the memory controller 12260. Thedecoder 12240 may perform the operation of the decoder 1600 describedwith reference to FIGS. 1 and 8 to 12 according to at least one exampleembodiment. That is, the decoder 12240 may read information about anencoding mode from the bitstream received from the memory controller12260 and may generate reconstruction image data based on the readencoding mode.

The application processor 12600 may apply various image processingtechniques based on image data received from the image sensor 12200. Theapplication processor 12600 may correspond to the application processor1800 of FIG. 1, but is not limited thereto, and thus, additionaldescription will be omitted to avoid redundancy.

FIG. 13 is a block diagram illustrating an image processing apparatusaccording to at least one example embodiment.

An image processing apparatus 13000 corresponds to at least one exampleembodiment, such as the image processing apparatus 1000 of FIG. 1.Accordingly, even though omitted below, the above description given withregard to the image processing apparatus 1000 of FIG. 1 may be appliedto the image processing apparatus 13000.

Referring to FIG. 13, the image processing apparatus 13000 may includean image sensor 13200, an MIPI (Mobile Industry Processor Interface)13400, and/or an application processor 13600, etc., but the exampleembodiments are not limited thereto. The image sensor 13200 is alsocalled a “2 stack sensor”.

The image sensor 13200 may include an image signal processor (ISP) 13220and an encoder 13240. The ISP 13220 may perform pre-processing onoriginal image data of a Bayer pattern. The pre-processing may meanapplying various image improvement algorithms which may improve thequality of image data. The pre-processed image data may be provided tothe encoder 13240.

The encoder 13240 may encode original image data of a Bayer patternreceived from the ISP 13220 and may output a bitstream including encodeddata. The encoder 13240 may perform the operation of the encoder 1400described with reference to FIGS. 1 to 7 according to at least oneexample embodiment, but is not limited thereto. That is, the encoder13240 may encode received original pixels based on the DPCM mode and theaverage mode and may generate a bitstream, etc. The bitstream generatedby the encoder 13240 may be provided to the MIPI 13400.

The MIPI 13400 which is an intermediate interface between the imagesensor 13200 and the application processor 13600 may be an electronicpart implemented with a circuit which may transmit an electrical signal.The MIPI 13400 may perform an interface operation for transmitting thereceived bitstream to the application processor 13600. The MIPI 13400may provide the bitstream received from the encoder 13240 to theapplication processor 13600. Accordingly, the image processing apparatus13000 may greatly increase the number of image frames which aretransmitted between the image sensor 13200 and the application processor13600 per second, by transmitting encoded data instead of transmittingoriginal data associated with image frames from the image sensor 13200to the application processor 13600.

The application processor 13600 may receive the bitstream including theencoded data from the MIPI 13400. The application processor 13600 mayinclude a decoder 13620 for decoding the received bitstream. That is,the decoder 13620 may read information about an encoding mode from thebitstream received from the MIPI 13400 and may generate reconstructionimage data based on the read encoding mode. The application processor13600 may apply various image processing techniques to thereconstruction image data.

An image processing apparatus according to at least one exampleembodiment of the inventive concepts may perform compression on originalBayer image. As the size of image data decreases through thecompression, efficiency associated with a memory space and a bandwidthof the image processing apparatus may be improved.

While the inventive concepts has been described with reference tovarious example embodiments thereof, it will be apparent to those ofordinary skill in the art that various changes and modifications may bemade thereto without departing from the spirit and scope of the exampleembodiments of the inventive concepts as set forth in the followingclaims.

What is claimed is:
 1. An image encoding apparatus comprising: at leastone processor configured to, receive one or more original pixels; andgenerate a compressed bitstream including encoded data corresponding tovalues of the one or more original pixels, based on an encoding modeselected from a plurality of encoding modes, the generating including,generating the compressed bitstream based on a difference value betweeneach of the values of the original pixels and a reference value, thereference value is based on at least one of the values of one or morereference pixels previously encoded and reconstructed, in response tothe encoding mode being a first mode of the plurality of encoding modes,generating the compressed bitstream based on an average value of atleast two of the values of the original pixels in response to theencoding mode being a second mode of the plurality of encoding modes,storing the difference value or the average value in a pixel region ofthe compressed bitstream, and performing a bit shift operation on thedifference value or the average value based on a size of the pixelregion.
 2. The image encoding apparatus of claim 1, wherein the at leastone processor is further configured to determine one of the first modeand the second mode as the encoding mode for the values of the one ormore original pixels; and the compressed bitstream includes a moderegion for storing information related to the determined encoding mode.3. The image encoding apparatus of claim 2, wherein the at least oneprocessor is further configured to obtain the values of the one or moreoriginal pixels using a Bayer color filter, the values of the one ormore original pixels including at least a value of a first pixel, avalue of a second pixel, a value of a third pixel, and a value of afourth pixel; and the first pixel and the second pixel are a greenpixel, and the third pixel and the fourth pixel are a red or blue pixel.4. The image encoding apparatus of claim 3, wherein, in the first mode,the at least one processor is configured to store a first differencevalue between the value of the first pixel and an average value of avalue of a first reference pixel and a value of a second referencepixel, and a second difference value between the value of the secondpixel and an average value of a value of a third reference pixel and avalue of a fourth reference pixel, in the pixel region; and the firstreference pixel and the second reference pixel are positioned on a upperline of the first pixel, and the third reference pixel and the fourthreference pixel are positioned on a upper line of the second pixel. 5.The image encoding apparatus of claim 3, wherein, in the first mode, theat least one processor is configured to store a third difference valuebetween a value of a fifth reference pixel and the value of the thirdpixel, and a fourth difference value between a value of a sixthreference pixel and the value of the fourth pixel, in the pixel region;and the fifth reference pixel is positioned on the second line above thethird pixel, and the sixth reference pixel is positioned on the secondline above the fourth pixel.
 6. The image encoding apparatus of claim 1,wherein the at least one processor is further configured to: detect abad pixel from the values of the one or more reference pixels, andwherein, in the first mode, a value of the bad pixel is not used as thereference value.
 7. The image encoding apparatus of claim 3, wherein, inthe second mode, the at least one processor is further configured tostore at least one of a first average value of the value of the firstpixel and the value of the second pixel, and a second average value ofthe value of the third pixel and the value of the fourth pixel, in thepixel region.
 8. The image encoding apparatus of claim 2, wherein the atleast one processor is further configured to: store informationregarding the number of times that the bit shift operation is performedin the mode region.
 9. The image encoding apparatus of claim 1, whereinthe at least one processor is further configured to determine one of thefirst mode and the second mode as the encoding mode for the values ofthe one or more original pixels based on whether a pixel of the originalpixels and the one or more reference pixels is included in an edgeregion of a Bayer color filter.
 10. An image decoding apparatuscomprising: at least one processor configured to, receive a bitstreamincluding, an encoding mode selected from a plurality of encoding modes,and a pixel region storing a difference value or an average value, thedifference value or the average value having been bit shifted a numberof times based on a size of the pixel region; and generate values of oneor more reconstruction pixels corresponding to values of one or moreoriginal pixels based on the received bitstream, the generatingincluding, decoding the bitstream based on the difference value includedin the bitstream and a reference value, the reference value is based onat least one value of one or more previously decoded reference pixels,in response to the encoding mode being a first mode of the plurality ofencoding modes, decoding the bitstream based on the average valuereceived from the bitstream in response to the encoding mode being asecond mode of the plurality of encoding modes, and the difference valueindicating a difference between each of the values of the one or moreoriginal pixels and the reference value, and the average value being avalue obtained by averaging at least two of the values of the one ormore original pixels.
 11. The image decoding apparatus of claim 10,wherein the bitstream includes a mode region for storing informationindicating one of the first mode and the second mode; and the at leastone processor is further configured to determine the encoding mode forthe values of the original pixels based on the mode region of thebitstream.
 12. The image decoding apparatus of claim 11, wherein the atleast one processor is further configured to obtain the values of theone or more original pixels using a Bayer color filter, the values ofthe one or more original pixels include at least a value of a firstpixel, a value of a second pixel, a value of a third pixel, and a valueof a fourth pixel; the first pixel and the second pixel are a greenpixel, and the third pixel and the fourth pixel are a red or blue pixel;and the one or more reconstruction pixels include a first reconstructionpixel corresponding to the first pixel, a second reconstruction pixelcorresponding to the second pixel, a third reconstruction pixelcorresponding to the third pixel, and a fourth reconstruction pixelcorresponding to the fourth pixel.
 13. The image decoding apparatus ofclaim 12, wherein in response to the determined encoding mode indicatingthe first mode, the at least one processor is configured to, determine avalue of the first reconstruction pixel based on a first differencevalue read from the pixel region and an average value of a value of afirst reference pixel and a value of a second reference pixel, anddetermine a value of the second reconstruction pixel based on a seconddifference value read from the pixel region and an average value of avalue of a third reference pixel and a value of a fourth referencepixel; and the first reference pixel and the second reference pixel arepositioned on a upper line of the first pixel, and the third referencepixel and the fourth reference pixel are positioned on a upper line ofthe second pixel.
 14. The image decoding apparatus of claim 13, whereinin response to the determined encoding mode indicating the first mode,the at least one processor is further configured to, determine a valueof the third reconstruction pixel based on a third difference value readfrom the pixel region and a fifth reference pixel, and determine a valueof the fourth reconstruction pixel based on a fourth difference valueread from the pixel region and a sixth reference pixel; and the fifthreference pixel is positioned on the second line above the third pixel,and the sixth reference pixel is positioned on the second line above thefourth pixel.
 15. The image decoding apparatus of claim 12, wherein inresponse to the determined encoding mode indicating the second mode, theat least one processor is further configured to: determine a firstaverage value read from the pixel region as the value of the first pixeland the value of the second pixel; and determine a second average valueread from the pixel region as the value of the third pixel and the valueof the fourth pixel.
 16. The image decoding apparatus of claim 11,wherein the at least one processor is further configured to: determine anumber of times to perform a bit shift operation on the difference valueor the average value read from the pixel region; perform the bit shiftoperation on the difference value or the average value read from thepixel region; and perform decoding based on the difference value or theaverage value obtained by performing the bit shift operation.
 17. Theimage decoding apparatus of claim 10, wherein the at least one processoris further configured to: detect a bad pixel from the values of currentreference pixels, and in response to the encoding mode indicating thefirst mode, not using a value of the bad pixel as the reference value.18. An image sensor connected with an memory comprising: an encoderconfigured to generate encoded data corresponding to values of originalpixels of a Bayer color pattern based on one of encoding modes; a memorycontroller configured to control an operation of inputting andoutputting the encoded data to and from the memory; and a decoderconfigured to generate values of reconstruction pixels corresponding tothe values of the original pixels by decoding the encoded data outputfrom the memory, wherein in a first mode of the encoding modes, theencoder is further configured to generate the encoded data based on adifference value between each of the values of the original pixels and areference value which is based on at least one of the values of thereference pixels previously encoded and reconstructed, in a second modeof the encoding modes, the encoder is further configured to generate theencoded data based on an average value of at least two of the values ofthe original pixels, and the encoder is further configured to store thedifference value or the average value in a pixel region of the generatedencoded data, and perform a bit shift operation on the difference valueor the average value based on a size of the pixel region.
 19. The imagesensor of claim 18, wherein the encoder determines one of the first modeand the second mode as an encoding mode for the values of the originalpixels based on whether a pixel of the original pixels and the referencepixels is included in an edge region of a whole image, and wherein theencoder includes information related to the determined encoding mode, inthe encoded data.
 20. The image sensor of claim 18, wherein the decoderdetermines one of the first mode and the second mode as an encoding modefor the values of the original pixels based on information parsed fromthe encoded data, and wherein the decoder generates the values of thereconstruction pixels based on the determined encoding mode.